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WWC Review of the Report “Does Working Memory Moderate the 
Effects of Fraction Intervention? An Aptitude-Treatment Interaction ” 1 

The findings from this review do not reflect the full body of research evidence on Fraction Face-Off ! 2 


What is this study about? 

The study examined the impacts of two versions of 
Fraction Face-Off!— one designed to build fluency 
(fluency version) in measurement interpretation topics 
(representing, comparing, and ordering fractions), and 
the other designed to solidify conceptual understand- 
ing (conceptual version) of the same measurement 
interpretation topics. The study assessed the impact 
of these versions of the program on the mathemat- 
ics achievement of fourth-grade students at risk for 
low mathematics achievement. 3 Fraction Face-Off! 
emphasizes the measurement approach to teaching 
fractions and the use of a number line to represent, 
compare, and order fractions. 

The study design is an individual-level randomized 
controlled trial. A total of 277 students at risk for 
low mathematics achievement from 49 classrooms 
in 14 schools— approximately half with more severe 
risk and half with less severe risk— were stratified by 
classroom and risk severity and randomly assigned 
to conditions: 94 to the fluency group, 91 to the 
conceptual group, and 92 to the comparison group. 4 
The analytic sample included a total of 243 stu- 
dents; 84 in the fluency group, 79 in the conceptual 
group, and 80 in the comparison group. The study 
authors examined each intervention group against the 
comparison group, and also compared the effect of 
the fluency version of Fraction Face-Off! to the con- 
ceptual version of Fraction Face-Off! The authors 
also compared the at-risk students in the Fractions 
Face-Off! conditions to a low-risk group of stu- 


dents to assess whether the intervention was able 
to reduce the achievement gap between the at-risk 
and low-risk students. 

Students in both intervention groups received three 
30-minute lessons per week for 12 weeks in small 
groups of three students with trained tutors. Both 
intervention groups received the same content 
during these small group sessions, focusing on the 
measurement approach to teaching fractions. The 
two intervention groups differed in the experiences 
they received during a 5-minute supplemental activ- 
ity period. During the supplemental period, students 
in the fluency group participated in strategic speed 
activities with flashcards designed to automate 
problem solving, while students in the conceptual 
group participated in activities with manipulatives 
designed to encourage reasoning about the prob- 
lem. The students in the comparison group received 
a similar amount of instruction using the regular dis- 
trict curriculum, enVisionMATH, which emphasizes 
the part-whole understanding of fractions. 

The study authors assessed impacts using selected 
released fraction items from the 1990-2009 
fourth- and eighth-grade National Assessment of 
Educational Progress (NAEP) math tests, and two 
researcher-developed measures— Fraction Number 
Line and Fraction Battery. 
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WWC Rating Features of Fraction Face-Off! 


The research described in this 
report meets WWC group design 
standards without reservations 

This study is a well-executed randomized controlled 
trial with low sample attrition for all three sets 
of comparisons examined through the random 
assignment design (Fraction Face-Off! fluency 
version versus comparison, Fraction Face-Off! 
conceptual version versus comparison, and Fraction 
Face-Off! fluency version versus Fraction Face-Off! 
conceptual version). The analyses that compared 
the at-risk students in the Fraction Face-Off! 
conditions to the low-risk group did not meet WWC 
group design standards because the groups were 
not equivalent at baseline. 

What did the study find? 

The study authors found, and the WWC confirmed, 
that both the fluency and conceptual versions of 
Fraction Face-Off! had a positive and statistically 
significant impact on all three outcome measures of 
the mathematics achievement of at-risk fourth-grade 
students. The study authors found, and the WWC 
confirmed, that there was no statistically significant 
difference between the fluency and conceptual 
groups on any of the three outcomes that measured 
mathematics achievement. 


Fraction Face-Off! is a math instruction program 
designed to improve knowledge of fractions 
and decimals in fourth-graders at risk for low 
mathematics achievement. The program, delivered 
individually or in a small group setting, consists of 
three 30-minute lessons per week for 12 weeks. The 
curriculum emphasizes a measurement approach 
to fractions which uses number lines, fraction tiles, 
and fraction circles to teach fraction concepts 
and calculations. Each lesson is structured to 
include teacher instruction, student group work, 
supplemental activities, and individual student work. 

In this study, two versions of Fractions Face- 
Off! were examined. The two versions were 
delivered by tutors in small groups and differed 
only in the 5-minute supplemental activities. The 
fluency groups participated in a sprint activity, 
which entailed collaborating to answer as many 
flashcard questions as possible during the allotted 
time, attempting to beat their score from the 
previous lesson. The activity was designed to 
automate problem solving. The conceptual groups 
demonstrated fractions with fraction circles or tiles, 
earning points for explaining their reasoning to the 
group and trying to beat the goal established by 
the tutor. The activity was designed to encourage 
reasoning about the problem. The activities for 
both conditions emphasized the measurement 
interpretation of fractions. To promote on-task 
behavior, tutors in both groups gave students 
rewards for on-task behavior and correct work in the 
study intervention groups. 
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Appendix A: Study details 

Fuchs, L., Schumacher, R., Sterba, S., Long, J., Namkung, J., Malone, A., Hamlett, C., Jordan, N., Gertsen, 
R., Siegler, R., & Changas, P. (2013). Does working memory moderate the effects of fraction interven- 
tion? An aptitude-treatment interaction. Journal of Educational Psychology, 106(2), 1-14. 

Setting The study was conducted in 49 classrooms in 14 schools (the location of the schools was not 
explicitly specified in the study). 

Study sample To identify fourth-grade students without an intellectual disability who were at risk for low 

mathematics achievement, the authors screened students using the Wide Range Achievement 
Test-4-Arithmetic (WRAT) and the two-subtest Wechsler Abbreviated Scales of Intelligence 
(WASI). Students scoring below the 35th percentile on the WRAT and at or above the 9th per- 
centile on both subtests of the WASI were eligible to participate in the study. From the eligible 
sample, the researchers stratified students by risk severity and randomly selected 2-8 at-risk 
students from each classroom, for a final sample of 277 fourth-grade students from 49 class- 
rooms in 14 schools— approximately half with more severe risk (below the 15th percentile on 
the WRAT) and half with less severe risk (between the 15th and 34th percentiles on the WRAT). 
The randomly selected students were then stratified by classroom and risk level and randomly 
assigned to conditions: 94 to the fluency group, 91 to the conceptual group, and 92 to the 
comparison group. The analytic sample included 243 students, including 84 students in the 
fluency group, 79 students in the conceptual group, and 80 students in the comparison group. 

In addition, the study also used a non-equivalent comparison group design to assess if the 
intervention was able to reduce the achievement gap between the at-risk students assigned to 
receive Fraction Face-Off! and a low-risk group of students. 

In the fluency group, the average WRAT score was 85.46 (SD = 6.29), and the average WASI 
score was 94.25 (SD = 12.20). Sixty-three percent of the fluency students were female, 14% 
were English learners, 93% were receiving subsidized lunch, 8% were receiving special educa- 
tion services, 58% were African American, 24% were Hispanic, 17% were White, and 1 % were 
categorized as other races. In the conceptual group, the average WRAT score was 85.52 (SD = 
6.74), and the average WASI score was 93.70 (SD = 1 1 .98). Sixty-two percent of the conceptual 
students were female, 14% were English learners, 95% were receiving subsidized lunch, 10% 
were receiving special education services, 61 % were African American, 22% were Hispanic, 

14% were White, and 3% were categorized as other races. In the comparison group, the aver- 
age WRAT score was 84.44 (SD = 7.1 0), and the average WASI score was 93.30 (SD = 1 1 .57). 
Fifty-nine percent of the comparison students were female, 5% were English learners, 86% were 
receiving subsidized lunch, 12% were receiving special education services, 58% were African 
American, 22% were Hispanic, 16% were White, and 4% were categorized as other races. 
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Intervention 

group 


Comparison 

group 


Outcomes and 
measurement 


Support for 
implementation 


Reason for 
review 


Students were pulled out in groups of three to receive the intervention during their math block, 
math center, or intervention period, depending on their teacher’s scheduling preferences. 

The lessons, delivered by a tutor, covered proper fractions; conversion of improper fractions 
to mixed numbers; and ordering, comparisons, and calculations with proper, improper, and 
mixed numbers. Concepts were illustrated with number lines, fraction lines, and fraction cir- 
cles. Each lesson consisted of four components, including teacher instruction, group work, a 
supplemental activity, and individual student work. Program content was the same for the flu- 
ency and conceptual groups, except for the supplemental activity period, which emphasized 
speedy problem solving with use of strategies for the fluency group and reasoning activities 
for the conceptual group. Students in the intervention groups did not practice the advanced 
skills of estimation and word problems, and the range of fractions covered in their sessions 
was limited. 

Students in the comparison group received the regular district math curriculum, enVison- 
MATH, which emphasizes the part-whole approach to fractions. The curriculum has six 
elements, including introduction of new concepts, practice on new concepts and skills, 
interactive discussions focusing on conceptual understanding, visual displays to support 
conceptual understanding, daily teacher assessment of student progress to understand indi- 
vidual needs, and differentiated instruction. In addition to classroom instruction, most at-risk 
students received instruction during the intervention period by a teacher in a small group (but 
groups were always larger than three students). The comparison curriculum included some 
advanced material, including estimation and word problems, and did not limit the range of 
fractions covered. 

The study authors administered released fraction items from the 1990-2009 NAEP, the Frac- 
tion Number Line, and Fraction Battery 201 1 -Revised assessments in September and October, 
before the intervention began in late October, and in early March, 2 weeks before the end of the 
intervention. For a more detailed description of these outcome measures, see Appendix B. 

Tutors participated in a week-long workshop training followed by bi-weekly 1-hour train- 
ings. The follow-up trainings focused on lesson content, student behavior management, and 
responding to varying student skill levels. The curriculum provided a manual with scripts for all 
36 lessons. Tutors were provided the opportunity to practice lesson delivery with other tutors 
before each new lesson. 

This study was identified for review by the WWC because it was supported by a grant to the 
University of Delaware (Principal Investigator: Nancy Jordan), with a subcontract to Vanderbilt 
University from the Institute of Education Sciences (IES). 
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Appendix B: Outcome measures for the mathematics achievement domain 


Mathematics achievement 


Mathematics achievement 


Fraction Battery 201 1 -Revised 

This researcher-developed measure included one subtest on fraction addition and one subtest on fraction sub- 
traction. Students received one point for providing the correct answer to questions that did not require students 
to reduce a fraction and two or three points for correctly reducing fractions one or two times, respectively, 
as required by the problem. The correlation of the two subtest scores was .81. A total score across the two 
subtests was computed for each student. The maximum score for the measure was 41 , and alpha for the study 
sample was .93. 

Fraction Number Line 

Fraction Number Line is a researcher-developed assessment. For this measure, students were presented with 
20 proper fractions, improper fractions, and mixed numbers to place on a number line with endpoints of 0 and 
2. For each item, a score was calculated as the difference between the student’s placement and the correct 
position of the fraction. Researchers computed the percent of absolute error score by taking the average of the 
20 item scores, dividing by two, and then multiplying by 100. The measure achieved test-retest reliability of .80 
on a sample of 63 students across 2 weeks. 

National Assessment of Educational 
Progress (NAEP)-selected items 

The study administered 19 released fraction items from the 1990-2009 NAEP math items, including easy, 
medium, or hard items from the fourth-grade assessment and easy items from eighth-grade assessment. 
Problems were read aloud to students. The assessment included eight items on part-whole interpretation and 
nine items on measurement interpretation of fractions. The maximum score for the measure was 25, and alpha 
for the study sample was .81 . 
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Appendix C: Study findings for the mathematics achievement domain 





Mean 

(standard deviation) 


WWC calculations 


Domain and 
outcome measure 

Study 

sample 

Sample 

size 

Intervention 

group 

Comparison 

group 

Mean 

difference 

Effect 

size 

Improvement 

index 

p-value 

Mathematics achievement 

Fraction Number Line 

Fluency vs. 
comparison 

14 schools/ 
164 students 

-0.20 

(0.08) 

-0.27 

(0.06) 

0.07 

0.98 

+34 

<.01 

NAEP-selected items 

Fluency vs. 
comparison 

14 schools/ 
164 students 

14.45 

(3.91) 

12.07 

(3.56) 

2.38 

0.63 

+24 

<.01 

Fraction Battery-Revised 

Fluency vs. 
comparison 

14 schools/ 
164 students 

17.53 

(7.83) 

8.16 

(4.71) 

9.37 

1.44 

+42 

<.01 

Domain average for fluency vs. comparison 




1.02 

+35 

Statistically 

significant 

Fraction Number Line 

Conceptual vs. 
comparison 

14 schools/ 
159 students 

-0.19 

(0.07) 

-0.27 

(0.06) 

0.08 

1.22 

+39 

<.01 

NAEP-selected items 

Conceptual vs. 
comparison 

14 schools/ 
159 students 

14.40 

(3.64) 

12.07 

(3.56) 

2.33 

0.64 

+24 

<.01 

Fraction Battery-Revised 

Conceptual vs. 
comparison 

14 schools/ 
159 students 

17.36 

(7.90) 

8.16 

(4.71) 

9.20 

1.41 

+42 

<.01 

Domain average for conceptual vs. comparison 




1.09 

+36 

Statistically 

significant 

Fraction Number Line 

Fluency vs. 
conceptual 

14 schools/ 
163 students 

-0.21 

(0.08) 

-0.21 

(0.08) 

-0.01 

-0.13 

-5 

> .05 

NAEP-selected items 

Fluency vs. 
conceptual 

14 schools/ 
163 students 

14.69 

(3.91) 

14.64 

(3.64) 

0.05 

0.01 

+1 

> .05 

Fraction Battery-Revised 

Fluency vs. 
conceptual 

14 schools/ 
163 students 

18.01 

(7.83) 

17.84 

(7.90) 

0.17 

0.02 

+1 

> .05 

Domain average for fluency vs. conceptual 




-0.03 

-1 

Statistically 

significant 


Table Notes: For mean difference, effect size, and improvement index values reported in the table, a positive number favors the intervention group and a negative number favors 
the comparison group. The signs of the means for the Fraction Number Line outcome were made negative so that a positive mean difference would reflect a favorable impact of 
the intervention. The effect size is a standardized measure of the effect of an intervention on individual outcomes, representing the average change expected for all individuals who 
are given the intervention (measured in standard deviations of the outcome measure). The improvement index is an alternate presentation of the effect size, reflecting the change 
in an average individual's percentile rank that can be expected if the individual is given the intervention. The WWC-computed average effect size is a simple average rounded to 
two decimal places; the average improvement index is calculated from the average effect size. The statistical significance of the study’s domain average was determined by the 
WWC. Some statistics may not sum as expected due to rounding. NAEP = National Assessment of Educational Progress. 

Study Notes: A correction for multiple comparisons was needed but did not affect whether any of the contrasts were found to be statistically significant. The p-values presented 
here were reported in the original study. The WWC calculated the program group mean using a difference-in-differences approach by adding the impact of the program (i.e., 
difference in mean gains between the intervention and comparison groups) to the unadjusted comparison group posttest means. Please see the WWC Procedures and Standards 
Handbook (version 3.0) for more information. The authors reported similar effect sizes for each of the three comparisons using a cross-classified partially nested model for each 
of the outcomes that accounted for nesting at the classroom level for all three conditions and at small group level for just the intervention conditions, while controlling for pretest 
scores. Effect size was calculated as an across-group mean difference divided by the SD within the comparison group only. For the fluency vs. comparison group, author-reported 
effect sizes were 0.99 for the Fraction Number Line, 0.60 for the NAEP, and 1.12 for the Fraction Battery. For the conceptual vs. comparison group, author-reported effect sizes 
were 0.80 for the Fraction Number Line, 0.63 for the NAEP, and 1.13 for the Fraction Battery. For the fluency vs. conceptual group, author-reported effect sizes were 0.24 for the 
Fraction Number Line, -0.03 for the NAEP, and -0.02 for the Fraction Battery. Both the fluency and conceptual versions of Fraction Face-Off! are characterized as having a statisti- 
cally significant positive effect because the effect for at least one measure within the domain is positive and statistically significant, and no effects are negative and statistically 
significant, accounting for multiple comparisons. The comparison of the fluency version of Fraction Face-Off! against the conceptual version of Fraction Face Off! is characterized 
as having an indeterminate effect because the mean effect reported is neither statistically significant nor substantively important. For more information, please refer to the WWC 
Standards and Procedures Handbook (version 3.0), pp. 25-26. 
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Endnotes 

1 Single study reviews examine evidence published in a study (supplemented, if necessary, by information obtained directly from the 
authors) to assess whether the study design meets WWC group design standards without reservations. The review reports the WWC’s 
assessment of whether the study meets WWC group design standards and summarizes the study findings following WWC conven- 
tions for reporting evidence on effectiveness. This study was reviewed using the Primary Mathematics protocol, version 3.0. 

2 A related intervention, Fraction Challenge, was the subject of a study examined in a previous single study review. See U.S. Depart- 
ment of Education, Institute of Education Sciences, What Works Clearinghouse. (2013, August). I N\NC review of the report: Improving 
at-risk learners’ understanding of fractions, available at http://whatworks.ed.gov. According to the authors, the two interventions differ 
primarily in that “ Fraction Face-Off! addresses a more ambitious set of skills, aligned with the Common Core State Standards” (p. 6). 

3 This study examined two research questions: (a) whether differences in working memory moderate the effects of the two versions 
of the intervention on student math achievement, and (b) whether the two versions of the intervention had effects on student math 
achievement. This review focuses only on the second research question because there are no WWC standards for examining the 
effect of a continuous moderator of an intervention. 

4 From the full population of students eligible to participate in the study, 2-8 students per classroom were randomly selected to be 
included in the study sample. For more detail on sample eligibility, see the description of the study sample in Appendix A. 

Recommended Citation 

U.S. Department of Education, Institute of Education Sciences, What Works Clearinghouse. (2014, December). 
WWC review of the report: Does working memory moderate the effects of fraction intervention? An 
aptitude-treatment interaction. Retrieved from http://whatworks.ed.gov 
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Glossary of Terms 

Attrition 


Clustering adjustment 
Confounding factor 

Design 
Domain 
Effect size 

Eligibility 

Equivalence 

Improvement index 


Multiple comparison 
adjustment 

Quasi-experimental 
design (QED) 

Randomized controlled 
trial (RCT) 

Single-case design 
(SCD) 

Standard deviation 


Statistical significance 
Substantively important 


Attrition occurs when an outcome variable is not available for all participants initially assigned 
to the intervention and comparison groups. The WWC considers the total attrition rate and 
the difference in attrition rates across groups within a study. 

If intervention assignment is made at a cluster level and the analysis is conducted at the individual 
level, the WWC will adjust the statistical significance to account for this mismatch, if necessary. 

A confounding factor is a component of a study that is completely aligned with one of the 
study conditions, making it impossible to separate how much of the observed effect was 
due to the intervention and how much was due to the factor. 

The design of a study is the method by which intervention and comparison groups were assigned. 
A domain is a group of closely related outcomes. 

The effect size is a measure of the magnitude of an effect. The WWC uses a standardized 
measure to facilitate comparisons across studies and outcomes. 

A study is eligible for review if it falls within the scope of the review protocol and uses either 
an experimental or matched comparison group design. 

A demonstration that the analysis sample groups are similar on observed characteristics 
defined in the review area protocol. 

Along a percentile distribution of individuals, the improvement index represents the gain 
or loss of the average individual due to the intervention. As the average individual starts at 
the 50th percentile, the measure ranges from -50 to +50. 

When a study includes multiple outcomes or comparison groups, the WWC will adjust 
the statistical significance to account for the multiple comparisons, if necessary. 

A quasi-experimental design (QED) is a research design in which study participants are 
assigned to intervention and comparison groups through a process that is not random. 

A randomized controlled trial (RCT) is an experiment in which eligible study participants are 
randomly assigned to intervention and comparison groups. 

A research approach in which an outcome variable is measured repeatedly within and 
across different conditions that are defined by the presence or absence of an intervention. 

The standard deviation of a measure shows how much variation exists across observations 
in the sample. A low standard deviation indicates that the observations in the sample tend 
to be very close to the mean; a high standard deviation indicates that the observations in 
the sample are spread out over a large range of values. 

Statistical significance is the probability that the difference between groups is a result of 
chance rather than a real difference between the groups. The WWC labels a finding statistically 
significant if the likelihood that the difference is due to chance is less than 5% (p < .05). 

A substantively important finding is one that has an effect size of 0.25 or greater, regardless 
of statistical significance. 


Please see the WWC Procedures and Standards Handbook (version 3.0) for additional details. 
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